Multi-task deep cross-attention networks for far-field speaker verification and keyword spotting

Abstract Personalized voice triggering is a key technology in assistants and serves as the first step for users to activate assistant. involves keyword spotting (KWS) speaker verification (SV). Conventional approaches this task include developing KWS SV systems separately. This paper proposes single system called multi-task deep cross-attention network (MTCANet) that simultaneously performs SV, while effectively utilizing information relevant both tasks. The proposed framework integrates sub-network an enhance performance challenging conditions such noisy environments, short-duration speech, model generalization. At core of MTCANet are three modules: novel (DCA) module integrate tasks, multi-layer stacked shared encoder (SE) reduce impact noise on recognition rate, soft attention (SA) modules allow focus pertinent middle layer preventing gradient vanishing. Our demonstrates outstanding well-off test set, improving by 0.2%, 0.023, 2.28% over well-known emphasized channel attention, propagation, aggregation time delay neural (ECAPA-TDNN) advanced Convmixer terms equal error rate (EER), minimum detection cost function (minDCF), accuracy (Acc), respectively.